Selecting content associated with a collection of entities

ABSTRACT

Methods, systems, and apparatus include computer programs encoded on a computer-readable storage medium, including a method for providing content. An indication is received from a content sponsor of a first collection of entities, sharing common characteristics, to be used for selection criteria for presenting a first content item. References to the first content item and selection criteria are stored. A query is received including terms or phrases associated with a first entity. It is determined that the first entity is included in the first collection of entities, wherein the terms or phrases do not include the first collection. Eligible content items are identified, each being associated with selection criteria including criteria specifying the first collection of entities. At least a portion of the first collection of entities is provided for presentation along with search results, including providing one or more of the eligible content items and the first content item.

CROSS-REFERENCE TO RELATED APPLICATION

This is a continuation of U.S. application Ser. No. 13/941,832, filed onJul. 15, 2013, the disclosure of which is considered part of and isincorporated by reference in the disclosure of this application.

BACKGROUND

This specification relates to information presentation.

The Internet provides access to a wide variety of resources. Forexample, video and/or audio files, as well as webpages for particularsubjects or particular news articles, are accessible over the Internet.Access to these resources presents opportunities for other content(e.g., advertisements) to be provided with the resources. For example, awebpage can include slots in which content can be presented. These slotscan be defined in the webpage or defined for presentation with awebpage, for example, along with search results.

Content slots can be allocated to content sponsors as part of areservation system, or in an auction. For example, content sponsors canprovide bids specifying amounts that the sponsors are respectivelywilling to pay for presentation of their content. In turn, an auctioncan be run, and the slots can be allocated to sponsors according, amongother things, to their bids and/or the relevance of the sponsoredcontent to content presented on a page hosting the slot or a requestthat is received for the sponsored content. The content can be providedto a user device such as a personal computer (PC), a smartphone, alaptop computer, a tablet computer, or some other user device. Contentcan be related in different ways, such as being related to one or moreentities or collections of entities.

SUMMARY

In general, one innovative aspect of the subject matter described inthis specification can be implemented in methods that include acomputer-implemented method for providing content. The method includesreceiving, from a content sponsor, an indication of a first collectionof entities to be used as selection criteria for presenting a firstcontent item of a campaign responsive to received requests, where thefirst collection of entities is a grouping of entities that share atleast one common characteristic. The method further includes storing ininventory a reference to the first content item and the indicatedselection criteria. The method further includes, after the storing,receiving a query including one or more terms or phrases, wherein theone or more terms or phrases identifies or is associated with a firstentity. The method further includes determining, using one or moreprocessors, that the first entity is included in the first collection ofentities, where the one or more terms or phrases does not include thefirst collection. The method further includes identifying one or moreeligible content items from the inventory of content items, eacheligible content item being associated with selection criteria includingcriteria specifying the first collection of entities, where identifyingincludes identifying the first content item. The method further includesproviding at least a portion of the first collection of entities forpresentation to a user along with search results responsive to thequery, where providing at least a portion of the first collection ofentities includes providing one or more of the eligible content itemsincluding the first content item along with the portion of the firstcollection of entities.

These and other implementations can each optionally include one or moreof the following features. The first collection of entities can benamed, and the method can further include providing to the contentsponsor an interface for enabling a selection of the first collection ofentities from a plurality of available collection of entities, andreceiving from content sponsor an explicit designation of the firstcollection of entities by name to be used as the selection criteria fordelivery of the first content item. The explicit designation can includea designation of the first collection of entities and an amount of a bidto be used when evaluating the first content item with other eligiblecontent items in an auction or reservation selection system. Determiningthat the first entity is included in the first collection of entitiescan include determining when the one or more terms or phrases indicatesa collection of entities and identifying the collection of entities asthe first collection of entities. Determining the first entity isincluded in the first collection of entities can include determiningwhen the one or more terms or phrases indicates a first entity, locatinga collection that includes the first entity and identifying thecollection that includes the first entity as the first collection ofentities. Determining the first entity is included in the firstcollection of entities can include determining when the one or moreterms or phrases indicates a collection of entities and identifying thecollection as the first collection of entities, or determining when theone or more terms or phrases indicate a first entity, locating acollection that includes the first entity and identifying the collectionthat includes the first entity as the first collection of entities. Themethod can further include determining a plurality of collections ofentities based at least in part on the one or more terms or phrases andselecting one of the plurality of collections of entities as the firstcollection of entities based at least in part on a predeterminedselection criteria. The predetermined selection criteria can be selectedfrom the group comprising a most popular collection, a most likelycollection or a most relevant collection. The first collection ofentities can include a plurality of elements that are displayable on adevice for presentation to a user, and the one or more eligible contentitem can be presented in a slot that precedes, succeeds or is interposedwith the elements of the first collection of entities. The one or moreeligible content items can be advertisements, and the first collectionof entities can be a group of entities that are distinct from the searchresults. The method can further include: presenting an interface to acontent sponsor for a campaign that is associated with distribution ofthe first content item, the interface including a control for indicatingone or more selection criteria for use in determining when to identifythe first content item as an eligible content item for consideration fordelivery in response to a received request for a content item; receivinga selection using the control of a collection, the collectionrepresenting a group of entities that share at least one commoncharacteristic, the collection designation specifying when to presentthe first content item responsive to requests including requiring atleast a portion of the collection be presented along with the firstcontent item; and storing the campaign including storing an associationbetween the selected collection and the first content item. The methodcan further include verifying that the first content item qualifies asbeing included in the first collection else disallowing the selection touse the first collection as selection criteria for the campaign.

In general, another innovative aspect of the subject matter described inthis specification can be implemented in computer program products thatinclude a computer program product tangibly embodied in acomputer-readable storage device and comprising instructions. Theinstructions, when executed by one or more processors, cause theprocessor to: receive, from a content sponsor, an indication of a firstcollection of entities to be used as a selection criteria for presentinga first content item of a campaign responsive to received requests,wherein the first collection of entities is a grouping of entities thatshare at least one common characteristic; store in inventory a referenceto the first content item and the indicated selection criteria; afterthe storing, receive a query including one or more terms or phrases,wherein the one or more terms or phrases identifies or is associatedwith a first entity; determine that the first entity is included in thefirst collection of entities, wherein the one or more terms of phrasesdoes not include the first collection; identify one or more eligiblecontent items from the inventory of content items, each eligible contentitem being associated with selection criteria including criteriaspecifying the first collection of entities, wherein identifyingincludes identifying the first content item; and provide at least aportion of the first collection of entities for presentation to a useralong with search results responsive to the query, wherein providing atleast a portion of the first collection of entities includes providingone or more of the eligible content items including the first contentitem along with the portion of the first collection of entities.

These and other implementations can each optionally include one or moreof the following features. The first collection of entities can benamed, and the instructions can further include instructions that causethe one or more processors to provide to the content sponsor aninterface for enabling a selection of the first collection of entitiesfrom a plurality of available collection of entities, and receive fromcontent sponsor an explicit designation of the first collection ofentities by name to be used as the selection criteria for delivery ofthe first content item. The explicit designation can include adesignation of the first collection of entities and an amount of a bidto be used when evaluating the first content item with other eligiblecontent items in an auction or reservation selection system. Determiningthat the first entity is included in the first collection of entitiescan include determining when the one or more terms or phrases indicatesa collection of entities and identifying the collection of entities asthe first collection of entities.

In general, another innovative aspect of the subject matter described inthis specification can be implemented in systems, including a systemcomprising one or more processors and one or more memory elementsincluding instructions. The instructions, when executed, cause the oneor more processors to: receive, from a content sponsor, an indication ofa first collection of entities to be used as a selection criteria forpresenting a first content item of a campaign responsive to receivedrequests, wherein the first collection of entities is a grouping ofentities that share at least one common characteristic; store ininventory a reference to the first content item and the indicatedselection criteria; after the storing, receive a query including one ormore terms or phrases, wherein the one or more terms or phrasesidentifies or is associated with a first entity; determine that thefirst entity is included in the first collection of entities, whereinthe one or more terms of phrases does not include the first collection;identify one or more eligible content items from the inventory ofcontent items, each eligible content item being associated withselection criteria including criteria specifying the first collection ofentities, wherein identifying includes identifying the first contentitem; and provide at least a portion of the first collection of entitiesfor presentation to a user along with search results responsive to thequery, wherein providing at least a portion of the first collection ofentities includes providing one or more of the eligible content itemsincluding the first content item along with the portion of the firstcollection of entities.

These and other implementations can each optionally include one or moreof the following features. The first collection of entities can benamed, and the instructions can further include instructions that causethe one or more processors to provide to the content sponsor aninterface for enabling a selection of the first collection of entitiesfrom a plurality of available collection of entities, and receive fromcontent sponsor an explicit designation of the first collection ofentities by name to be used as the selection criteria for delivery ofthe first content item. The explicit designation can include adesignation of the first collection of entities and an amount of a bidto be used when evaluating the first content item with other eligiblecontent items in an auction or reservation selection system. Determiningthat the first entity is included in the first collection of entitiescan include determining when the one or more terms or phrases indicatesa collection of entities and identifying the collection of entities asthe first collection of entities.

Particular implementations may realize none, one or more of thefollowing advantages. Content sponsors can specify that their contentitems are to be included (or presented more prominently) when acollection of entities is presented to a user.

The details of one or more implementations of the subject matterdescribed in this specification are set forth in the accompanyingdrawings and the description below. Other features, aspects, andadvantages of the subject matter will become apparent from thedescription, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of an example environment for deliveringcontent.

FIG. 2 shows an example system for providing a collection of entitieswith search results.

FIG. 3 is a flowchart of an example process for providing a collectionof entities with search results.

FIG. 4 is a block diagram of an example computer system that can be usedto implement the methods, systems and processes described in thisdisclosure.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

This document describes systems, methods, computer program products andmechanisms for providing content in association with a collection ofentities, e.g., in a display to a user. For example, an indication canbe received, from a content sponsor, of a first collection of entitiesto be used as selection criteria for presenting a first content item ofa campaign responsive to received requests. The first collection ofentities is a grouping of entities that share at least one commoncharacteristic. A reference to the first content item and the indicatedselection criteria is stored in inventory. After the storing, a query isreceived including one or more terms or phrases, where the one or moreterms or phrases identifies or is associated with a first entity. It isdetermined that the first entity is included in the first collection ofentities, wherein the one or more terms or phrases do not include thefirst collection. One or more eligible content items are identified froman inventory of content items. Each eligible content item is associatedwith selection criteria including criteria specifying the firstcollection of entities, wherein identifying includes identifying thefirst content item. At least a portion of the first collection ofentities is presented for presentation to a user along with searchresults responsive to the query, wherein providing at least a portion ofthe first collection of entities includes providing one or more of theeligible content items including the first content item along with theportion of the first collection of entities.

In some implementations, the collection of entities can be providedalong with search results in response to a query entered by a user in abrowser. One or more content items related to the collection of entitiescan also be identified. The user can be presented with search resultsresponsive to the query, e.g., in one area of a user interface. Thecollection of entities can be displayed in a different area of the userinterface, and the collection of entities can include one or morecontent items that are related to the collection of entities. Othermechanisms can be used for selecting and presenting content that isassociated with collections of entities.

FIG. 1 is a block diagram of an example environment 100 for deliveringcontent. The example environment 100 includes a content managementsystem 110 for selecting and providing content in response to requestsfor content. The example environment 100 includes a network 102, such asa local area network (LAN), a wide area network (WAN), the Internet, ora combination thereof. The network 102 connects websites 104, userdevices 106, content sponsors 108 (e.g., advertisers), publishers 109,and the content management system 110. The example environment 100 mayinclude many thousands of websites 104, user devices 106, contentsponsors 108 and publishers 109.

In some implementations, in response to a request for content, thecontent management system 110 can provide a collection of entities andsearch results 134 that is responsive to the request. For example, thesearch results can be displayable to the user in a search results areaof a browser or web page. The collection of entities can be displayed,for example, in a knowledge graph area or some other area separate fromthe search results area (and including different content). In someimplementations, the collection of entities can include organic content,e.g., entity-related advertisements that are all related in some way,such as by at least one common characteristic. The collection ofentities can also include a related content item that is added to thecollection of entities presented to the user. The content item can be,for example, an advertisement that an associated content sponsor hasspecified to be selected when related to a collection of entities.

The content management system 110 can include plural engines. An entityengine 121, for example, can evaluate terms and phrases in a queryreceived by the content management system 110 to identify at least oneassociated collection of entities. A collection of entities identifiedby the entity engine 121 can include entities that share at least onecommon characteristic. Entities can include, for example, one or more ofa product, a product category, a product manufacturer, a specific modelof a product, a vertical, a service, a service category, or a useractivity. The entity engine 121 can select one or more particularcollections of entities, e.g., that are the most likely collections ofentities to be associated with the terms and phrases. In someimplementations, the entity engine can select one collection of entitiesfor evaluation and presentation to a user.

A related content identification engine 122, for example, can identifyeligible content items from an inventory of content items (e.g.,eligible content items 132). Each eligible content item that isidentified can be associated with selection criteria, including criteriaspecifying (or associated with) a specific collection of entities and areceived query.

A request handler 123, for example, can handle requests for contentreceived by the content management system 110, and in response to eachrequest, provide one or more content items to the requestor. Forexample, the request for content can be a request to fill a content itemslot on a web page displayed in a browser on the user device 106. Inresponse to the request for content, the request handler 123 can providea content item, e.g., an advertisement having characteristics that matchthe characteristics of an advertisement slot on the user device 106. Therequest handler 123 can also provide search results 118 in response toreceived search queries 116.

The environment 100 can include plural data stores, which can be storedlocally by the content management system 110, stored somewhere else andaccessible using the network 102, generated as needed from various datasources, or some combination. A data store of entities 131, for example,can include identifiers, definitions and other information aboutentities such as products, product categories, product manufacturers,specific models of products, verticals, services, service categories oruser activities. Entities 131 can also include information for use inidentifying entities, e.g., including keywords or other informationassociated with each entity that can be used by the entity engine 121for identifying entities and collections of entities associated with aparticular content item. Entities and entity definitions can behierarchical, meaning that one entity (e.g., cameras) can include othersub-entities (e.g., land cameras and digital cameras). Otherrelationships among entities can exist. For example, entities can becommercially related as determined by evaluating one or more of purchasehistory or other resource data (such as ad inventory data).

The eligible content items 132 can include content items, such asadvertisements, that can be selected based on selected criteria. Forexample, advertisements that are included in the eligible content items132 can be selected using keywords and/or other selection criteria thatcan correspond to entities.

In some implementations, content sponsors 108 can be provided with aninterface (e.g., a content sponsor interface) by which a particularcontent sponsor 108 can specify campaign parameters for selecting theircontent. One possible selection criteria that can be specified can be acollection of entities. For example, for a given advertising campaign,the content sponsor 108 can specify that an advertisement is to beprovided with selected collections of entities when they are presentedto a user. In some implementations, the parameters can specify, forexample, that the advertisement is to be displayed or presented firstwithin the collection of entities. In some implementations, theparameters can specify, for example, that when an advertisementassociated with a given campaign is already to be provided in adesignated collection, that the advertisement be placed in a moreprominent location (e.g., moved to the beginning of a list for immediatedisplay without requiring scrolling or paging). The moved or highlightedadvertisement can obviate the inclusion of the same content at thelesser prominent location in the collection presentation. In someimplementations, the content sponsor 108 can specify that a differentversion of a creative is to be used when being selected based oncollections of entities. Collections of entities and presentation ofother content items with collections of entities is discussed in greaterdetail below with respect to FIG. 2.

A website 104 includes one or more resources 105 associated with adomain name and hosted by one or more servers. An example website is acollection of webpages formatted in hypertext markup language (HTML)that can contain text, images, multimedia content, and programmingelements, such as scripts. Each website 104 can be maintained by acontent publisher, which is an entity that controls, manages and/or ownsthe website 104.

A resource 105 can be any data that can be provided over the network102. A resource 105 can be identified by a resource address that isassociated with the resource 105. Resources include HTML pages, wordprocessing documents, portable document format (PDF) documents, images,video, and news feed sources, to name only a few. The resources caninclude content, such as words, phrases, images, video and sounds, thatmay include embedded information (such as meta-information hyperlinks)and/or embedded instructions (such as JavaScript™ scripts).

A user device 106 is an electronic device that is under control of auser and is capable of requesting and receiving resources over thenetwork 102. Example user devices 106 include personal computers (PCs),televisions with one or more processors embedded therein or coupledthereto, set-top boxes, mobile communication devices (e.g.,smartphones), tablet computers and other devices that can send andreceive data over the network 102. A user device 106 typically includesone or more user applications, such as a web browser, to facilitate thesending and receiving of data over the network 102.

The user device 106 can request resources 105 from a website 104. Inturn, data representing the resource 105 can be provided to the userdevice 106 for presentation by the user device 106. The datarepresenting the resource 105 can also include data specifying a portionof the resource or a portion of a user display, such as a presentationlocation of a pop-up window or a slot of a third-party content site orwebpage, in which content can be presented. These specified portions ofthe resource or user display are referred to as slots (e.g., ad slots).

To facilitate searching of these resources, the environment 100 caninclude a search system 112 that identifies the resources by crawlingand indexing the resources provided by the content publishers on thewebsites 104. Data about the resources can be indexed based on theresource to which the data corresponds. The indexed and, optionally,cached copies of the resources can be stored in an indexed cache 114.

User devices 106 can submit search queries 116 to the search system 112over the network 102. In response, the search system 112 can, forexample, access the indexed cache 114 to identify resources that arerelevant to the search query 116. The search system 112 identifies theresources in the form of search results 118 and returns the searchresults 118 to the user devices 106 in search results pages. A searchresult 118 can be data generated by the search system 112 thatidentifies a resource that is provided in response to a particularsearch query, and includes a link to the resource. In someimplementations, the search results 118 include the content itself, suchas a map, or an answer, such as in response to a query for a store'sproducts, phone number, address or hours of operation. In someimplementations, the content management system 110 can generate searchresults 118 using information (e.g., identified resources) received fromthe search system 112. An example search result 118 can include awebpage title, a snippet of text or a portion of an image extracted fromthe webpage, and the URL of the webpage. Search results pages can alsoinclude one or more slots in which other content items (e.g., ads) canbe presented. In some implementations, slots on search results pages orother webpages can include content slots for content items that havebeen provided as part of a reservation process. In a reservationprocess, a publisher and a content item sponsor enter into an agreementwhere the publisher agrees to publish a given content item (or campaign)in accordance with a schedule (e.g., provide 1000 impressions by date X)or other publication criteria. In some implementations, content itemsthat are selected to fill the requests for content slots can be selectedbased, at least in part, on priorities associated with a reservationprocess (e.g., based on urgency to fulfill a reservation).

When a resource 105, search results 118 and/or other content arerequested by a user device 106, the content management system 110receives a request for content. The request for content can includecharacteristics of the slots that are defined for the requested resourceor search results page, and can be provided to the content managementsystem 110.

For example, a reference (e.g., URL) to the resource for which the slotis defined, a size of the slot, and/or media types that are availablefor presentation in the slot can be provided to the content managementsystem 110 in association with a given request. Similarly, keywordsassociated with a requested resource (“resource keywords”) or a searchquery 116 for which search results are requested can also be provided tothe content management system 110 to facilitate identification ofcontent that is relevant to the resource or search query 116.

Based at least in part on data included in the request, the contentmanagement system 110 can select content that is eligible to be providedin response to the request (“eligible content items”). For example,eligible content items can include eligible ads having characteristicsmatching the characteristics of ad slots and that are identified asrelevant to specified resource keywords or search queries 116. In someimplementations, the selection of the eligible content items can furtherdepend on user signals, such as demographic signals and behavioralsignals.

The content management system 110 can select from the eligible contentitems that are to be provided for presentation in slots of a resource orsearch results page based at least in part on results of an auction (orby some other selection process). For example, for the eligible contentitems, the content management system 110 can receive offers from contentsponsors 108 and allocate the slots, based at least in part on thereceived offers (e.g., based on the highest bidders at the conclusion ofthe auction or based on other criteria, such as those related tosatisfying open reservations). The offers represent the amounts that thecontent sponsors are willing to pay for presentation (or selection orother interaction with) of their content with a resource or searchresults page. For example, an offer can specify an amount that a contentsponsor is willing to pay for each 1000 impressions (i.e.,presentations) of the content item, referred to as a CPM bid.Alternatively, the offer can specify an amount that the content sponsoris willing to pay (e.g., a cost per engagement) for a selection (i.e., aclick-through) of the content item or a conversion following selectionof the content item. For example, the selected content item can bedetermined based on the offers alone, or based on the offers of eachcontent sponsor being multiplied by one or more factors, such as qualityscores derived from content performance, landing page scores, and/orother factors.

A conversion can be said to occur when a user performs a particulartransaction or action related to a content item provided with a resourceor search results page. What constitutes a conversion may vary fromcase-to-case and can be determined in a variety of ways. For example, aconversion may occur when a user clicks on a content item (e.g., an ad),is referred to a webpage, and consummates a purchase there beforeleaving that webpage. A conversion can also be defined by a contentprovider to be any measurable or observable user action, such asdownloading a white paper, navigating to at least a given depth of awebsite, viewing at least a certain number of webpages, spending atleast a predetermined amount of time on a web site or webpage,registering on a website, experiencing media, or performing a socialaction regarding a content item (e.g., an ad), such as republishing orsharing the content item. Other actions that constitute a conversion canalso be used.

In some implementations, conversions may be more likely to occur whenthe content item is displayed among a collection of entities, e.g., as afirst entity (or first page instance) among a collection of entities.For example, the user may be more likely to interact with amovie-related advertisement or some other content item if it presentedas the first advertisement among others that are related, e.g., by beingassociated with and/or being compared with other entities in thecollection of entities.

FIG. 2 shows an example system 200 for providing a collection ofentities 202 with search results 204. For example, the contentmanagement system 110 can provide the search results 204 in response toa search query 206 received from the user device 106. At the same time,the content management system 110 can provide the collection of entities202 (e.g., advertisements) that can include, for example, a content itemthat is identified using the search query 206 selection criteria used inidentifying the entities. A more detailed example follows using anexample sequence of stages 1-4.

At stage 1, the content management system 110 can receive a query 208that includes one or more terms and/or phrases. The query can be based,for example, on the search query 206, e.g., “best movie 1996” that auser 210 has entered on a web page 212. The query 208 can also includeother information, e.g., profile information of the user, the user'slocation, and/or other information not included in the search query 206.The search query 206, for example, can be a query that a user 210 hasentered on a web page 212.

In some implementations, the query 208 can include information thatsignals the content management system 110, for example, that collectionsof entities are to be considered in processing the query. For example,the information can be based on user settings and/or preferences inwhich the user has indicated an interest in receiving more pertinentcontent based on entities and collections of entities.

At stage 2, the entity engine 121, for example, can determine a firstcollection of entities 214 based at least in part on the one or moreterms or phrases included in the query 208. The first collection ofentities 214 can be, for example, a grouping of entities (e.g., movies,or advertisements about movies) that share at least one commoncharacteristic (e.g., related to a movie or television show or otherproduct, service or event). The entity engine 121 can determine thecommon characteristic, for example, using terms and/or phrases in “bestmovie 1996.” In the current example, entities 214 a-214 c are identifiedthat are in a collection of movies determined using the query 208.

At stage 3, the related content identification engine 122, for example,can identify eligible content items 216 from an inventory of contentitems (e.g., eligible content items 132). Each eligible content itemthat is identified can be associated with selection criteria, includingcriteria specifying (or associated with) the first collection ofentities 214. For example, the selection criteria can include selectioncriteria related to the best movie of 1996, movies in general, actorsand actresses, the year 1996, and/or other selection criteria.

At stage 4, the content management system 110, for example, can providea combined result 218 for presentation to the user 210 that includes atleast a portion of the first collection of entities 214 along withsearch results 204. For example, the portion of the first collection ofentities can include one or more of the eligible content items (e.g., anadvertisement for Movie 4) along with the portion (e.g., some or all) ofthe first collection of entities 214. In this example, the advertisementfor Movie 4 is associated with an entity 216 a, e.g., associated withMovie 4.

The received collection of entities and search results can be displayedon the web page 212 in different ways. For example, the web page 212 canbe organized such that search results 204 responsive to the search query206 are displayed in one area. Other areas of the web page 212 can beused to display additional information, e.g., in a knowledge graph area220. In some implementations, the knowledge graph area 220 can includeimages 222 (e.g., with images 222 a-222 c related to the firstcollection of entities 214), header information 224 (e.g., identifyingat least a title and general information), and cast information 226(e.g., including cast member images 226 a-226 c), and other information.

In some implementations, the knowledge graph area 220 can also include acollection area 228, e.g., an area in which the collection of entities202 is displayed. For example, the collection of entities 202 caninclude entities 202 a-202 c that correspond to the first collection ofentities 214 identified by the entity engine 121. An entity 202 d, forexample, can be (or correspond to) the advertisement for Movie 4 asidentified by the related content identification engine 122.

In some implementations, the presentation order of the collection ofentities 202 can depend on various factors. For example, contentsponsors can be allowed to bid on the selection of their content items(e.g., advertisements) to be presented with collections of entities. Insome implementations, content sponsors can specify that their contentitems, when selected, are to be presented more prominently, e.g., listedfirst (e.g., before the entities 202 a). In another example, thepresentation order/location of a content sponsor's content item can beimproved, e.g., so that the entity 202 d is guaranteed to appear on afirst screen when the collection of entities 202 is a scrollable list.Other means of accentuating or otherwise emphasizing a specific piece ofcontent in the collection are possible.

The collection of entities 202 is just one example of sponsored contentthat can be displayed in the collection area 228 or other areas on theweb page 212. In another example, other types of sponsored content(e.g., advertisements for hotels and restaurants) can be displayed,e.g., in a task-triggered event, such as if it is suspected that theuser is searching for vacation information related to a particulardestination location.

FIG. 3 is a flowchart of an example process 300 for providing acollection of entities with search results. In some implementations, thecontent management system 110 can perform stages of the process 300using instructions that are executed by one or more processors. FIGS.1-2 are used to provide example structures for performing the stages ofthe process 300.

An indication is received from a content sponsor of a first collectionof entities to be used as selection criteria for presenting a firstcontent item of a campaign responsive to received requests (302). Thefirst collection of entities is a grouping of entities that share atleast one common characteristic. For example, as part of a campaign, acontent sponsor 108 can use a content sponsor interface to specify thatcertain entities 131 are to be used in association with the selection ofeligible content items 132 in response to requests for content receivedfrom users.

A reference to the first content item and the indicated selectioncriteria are stored in inventory (304). For example, the informationabout the entities and eligible content items associated with thecontent sponsor's campaign can be stored, e.g., in a data store ofcampaigns.

After the storing, a query is received including one or more terms orphrases, where the one or more terms or phrases identifies or isassociated with a first entity (306). For example, the contentmanagement system 110 can receive the query 208 from the user device106. The query 208 can include terms and phrases, e.g., “best movie1996” entered by the user 210 on the web page 212, e.g., that isassociated with one of the entities 131 related to movies. As anexample, the entity engine 121 can determine the first collection ofentities 214 that can include entities 214 a-214 c corresponding toMovies 1-3, respectively, are associated with the query 208.

In some implementations, the first collection of entities can be named.The process 300 can further include providing to the content sponsor aninterface for enabling a selection of the first collection of entitiesfrom a plurality of available collection of entities, and receiving fromcontent sponsor an explicit designation of the first collection ofentities by name to be used as the selection criteria for delivery ofthe first content item. For example, the collection of entities thatincludes entities 214 a-214 c can have a user-friendly name such as“1996 Movies that are best rated,” “Best rated related Entities” or someother descriptive name. In another example, the name of a collection ofentities can be represented by a number, a path name, and/or some otheridentifier or mechanism. Any or all of these names for the collection ofentities can have associated with them an explicit list of entities thatbelong to the collection (e.g., a list of movie titles). Using a contentsponsor interface, for example, a content sponsor can select one or moreentities from a list of presented entities. The selected entities can bestored, e.g., with other campaign information associated with thecampaign sponsor.

In some implementations, the explicit designation can include adesignation of the first collection of entities and an amount of a bidto be used when evaluating the first content item with other eligiblecontent items in an auction or reservation selection system. Forexample, bids provided by content sponsors for the presentation of theirsponsored content can be tied to specific names of collections ofentities and corresponding bid amounts to be used in an auction withother eligible content items.

In some implementations, determining that the first entity is includedin the first collection of entities can include determining when the oneor more terms or phrases indicates a collection of entities andidentifying the collection entities as the first collection of entities.For example, to determine the collection of entities 214, the entityengine 121 can use terms or phrases in the query 208 to look up (e.g.,in a table, data base or other structure) the corresponding associatedentities, such as the collection of entities that includes entities 214a-214 c. In some implementations, information used for this type ofdetermination can be generated offline, e.g., from historical queries ofusers, using the queries terms and phrases they used and thecorresponding entities represented by the responsive content. Other wayscan be used for using historical information for associating terms andphrases with entities and/or collections of entities.

In some implementations, determining the first entity is included in thefirst collection of entities can include determining when the one ormore terms or phrases indicates a first entity, locating a collectionthat includes the first entity, and identifying the collection thatincludes the first entity as the first collection of entities. Forexample, the entity engine 121 can use one or more terms or phrases(e.g., “Best movie 1996” in the query 208 to determine an entity (e.g.,Movie 1). The entity engine 121 can then use the entity to look up acollection of entities that includes the entity (e.g., all entities thatinclude Movie 1 as part of a collection).

In some implementations, determining the first entity is included in thefirst collection of entities can be performed in either one of at leasttwo ways. In some implementations, it is determined when the one or moreterms or phrases indicates a collection of entities, and the collectionis identified as the first collection of entities. For example, theentity engine 121 can use one or more terms in “best movies 1996” tolook up a corresponding collection of entities (e.g., the criticallyacclaimed movies of 1996), and identify that collection of entities asthe first collection of entities 214. In some implementations, it isdetermined when the one or more terms or phrases indicate a firstentity, a collection is located that includes the first entity, and thecollection is identified that includes the first entity as the firstcollection of entities. Other methods can be used for determining thefirst collection of entities.

In some implementations, the process 300 can further include steps forselecting the first collection of entities from a plurality ofcollections of entities. A plurality of collections of entities isdetermined based at least in part on the one or more terms or phrases.For example, the entity engine 121 can use information from the receivedquery 208 to determine multiple collections of entities associated withthe query 208, e.g., including collections of movies and/or collectionsof other entities. One of the plurality of collections of entities isselected as the first collection of entities based at least in part on,for example, a predetermined selection criteria. For example, the entityengine 121 can use predetermined selection criteria such as a mostpopular collection, a most likely collection, or a most relevantcollection to select one of the plurality of collections as the firstcollection of entities 214.

In some implementations, the first collection of entities can include aplurality of elements that are displayable on a device for presentationto a user, and the one or more eligible content item can be presented ina slot that precedes, succeeds or is interposed with the elements of thefirst collection of entities. As an example, the collection of entities214 can include entities 214 a-214 c that correspond to entities 202a-202 c that are presented in the collection area 228. The advertisementfor Movie 4 identified by the related content identification engine 122can be presented in the collection area 228, e.g., as the fourth elementas entity 202 d, as the associated entity 216 a related to Movie 4. Inthis example, the entity 202 d succeeds the entities 202 a-202 c. Insome implementations, the entity 202 d can precede (or be insertedwithin) the other entities 202 a-202 c. The placement can be based, atleast in part, on selection parameters specified by content providersfor the selection of their content, including bid amounts.

It is determined that the first entity is included in the firstcollection of entities, wherein the one or more terms or phrases do notinclude the first collection (308). For example, the entity engine 121can determine that the entity 216 a is included in the first collectionof entities 214 and that the query 208 does not include the firstcollection of entities 214.

In some implementations, the one or more eligible content items can beadvertisements, and the first collection of entities can be a group ofentities that are distinct from the search results. In the currentexample, the identified eligible content item 216 can be theadvertisement for the Movie 4 that is associated with the entity 216 a.Further, first collection of entities 214 are different from the searchresults 204.

One or more eligible content items are identified from an inventory ofcontent items (310). For example, each eligible content item can beassociated with selection criteria including criteria specifying thefirst collection of entities, and identifying the one or more eligiblecontent items can include identifying the first content item.

At least a portion of the first collection of entities is provided forpresentation to a user along with search results responsive to the query(312). Providing at least a portion of the first collection of entitiesincludes providing one or more of the eligible content items includingthe first content item along with the portion of the first collection ofentities. For example, the user can be presented with search results204, e.g., in a search results area of the web page. At the same time,the collection of entities 202 can be provided within the knowledgegraph area 220 or some other area separate from the search results 204.The collection of entities 202 that are presented can include most orall of the entities 214 a-214 c and at least one of the eligible contentitems 216. For example, presentation of the entity 202 d with theentities 202 a-202 c can cause some of the entities 202 not to be shown.One or more of the entities 202, for example, can be replaced by one ormore advertisements (e.g., entity 202 d) that are from the eligiblecontent items 216.

In some implementations, sponsored content can replace all of thecollection of entities 202 that would otherwise be displayed. Forexample, the collection of entities 202 can include only advertisementsthat are selected from the identified eligible content items 216 andexclude advertisements selected from the first collection of entities214.

In some implementations, the process 300 can further include additionalsteps that are used by a content sponsor, for example, to specifycampaign parameters for selecting and presenting content items withcollections of entities. For example, the content sponsor can bepresented with an interface for a campaign (e.g., an advertisementcampaign) that is associated with distribution of the first content item(e.g., an advertisement). The interface can include a control that thecontent sponsor can use, for example, to indicate one or more selectioncriteria for identifying when to serve the content (e.g.,advertisements). For example, the selection criteria can be used indetermining when to identify the first content item as an eligiblecontent item for consideration for delivery in response to a receivedrequest (e.g., a query) for a content item. The interface can receive,from the content sponsor using the control, a selection of a collection.The collection identification can represent a group of entities thatshare at least one common characteristic. When designating thecollection, the content sponsor can specify when to present the firstcontent item responsive to requests, including requiring that at least aportion of the collection be presented along with the first content item(e.g., when the collection is already scheduled to be presented to auser, present this content item associated with this campaign along withthe collection or in a prominent way in the collection). The campaigncan be stored, including storing an association between the selectedcollection and the first content item.

In some implementations, the content delivery system can verify that thefirst content item qualifies as being included in the first collection,and otherwise disallow the selection (e.g., disallow the user to use thefirst collection as selection criteria for the campaign). For example,if the content sponsor specifies an advertisement (e.g., for kitchenappliances) that is not related to a collection of entities (e.g.,action movies), then the interface can inform the content sponsor thatthe association is not allowed.

FIG. 4 is a block diagram of example computing devices 400, 450 that maybe used to implement the systems and methods described in this document,as either a client or as a server or plurality of servers. Computingdevice 400 is intended to represent various forms of digital computers,such as laptops, desktops, workstations, personal digital assistants,servers, blade servers, mainframes, and other appropriate computers.Computing device 400 is further intended to represent any othertypically non-mobile devices, such as televisions or other electronicdevices with one or more processers embedded therein or attachedthereto. Computing device 450 is intended to represent various forms ofmobile devices, such as personal digital assistants, cellulartelephones, smartphones, and other computing devices. The componentsshown here, their connections and relationships, and their functions,are meant to be examples only, and are not meant to limitimplementations of the inventions described and/or claimed in thisdocument.

Computing device 400 includes a processor 402, memory 404, a storagedevice 406, a high-speed controller 408 connecting to memory 404 andhigh-speed expansion ports 410, and a low-speed controller 412connecting to low-speed bus 414 and storage device 406. Each of thecomponents 402, 404, 406, 408, 410, and 412, are interconnected usingvarious busses, and may be mounted on a common motherboard or in othermanners as appropriate. The processor 402 can process instructions forexecution within the computing device 400, including instructions storedin the memory 404 or on the storage device 406 to display graphicalinformation for a GUI on an external input/output device, such asdisplay 416 coupled to high-speed controller 408. In otherimplementations, multiple processors and/or multiple buses may be used,as appropriate, along with multiple memories and types of memory. Also,multiple computing devices 400 may be connected, with each deviceproviding portions of the necessary operations (e.g., as a server bank,a group of blade servers, or a multi-processor system).

The memory 404 stores information within the computing device 400. Inone implementation, the memory 404 is a computer-readable medium. In oneimplementation, the memory 404 is a volatile memory unit or units. Inanother implementation, the memory 404 is a non-volatile memory unit orunits.

The storage device 406 is capable of providing mass storage for thecomputing device 400. In one implementation, the storage device 406 is acomputer-readable medium. In various different implementations, thestorage device 406 may be a floppy disk device, a hard disk device, anoptical disk device, or a tape device, a flash memory or other similarsolid state memory device, or an array of devices, including devices ina storage area network or other configurations. In one implementation, acomputer program product is tangibly embodied in an information carrier.The computer program product contains instructions that, when executed,perform one or more methods, such as those described above. Theinformation carrier is a computer- or machine-readable medium, such asthe memory 404, the storage device 406, or memory on processor 402.

The high-speed controller 408 manages bandwidth-intensive operations forthe computing device 400, while the low-speed controller 412 manageslower bandwidth-intensive operations. Such allocation of duties is anexample only. In one implementation, the high-speed controller 408 iscoupled to memory 404, display 416 (e.g., through a graphics processoror accelerator), and to high-speed expansion ports 410, which may acceptvarious expansion cards (not shown). In the implementation, low-speedcontroller 412 is coupled to storage device 406 and low-speed bus 414.The low-speed bus 414 (e.g., a low-speed expansion port), which mayinclude various communication ports (e.g., USB, Bluetooth®, Ethernet,wireless Ethernet), may be coupled to one or more input/output devices,such as a keyboard, a pointing device, a scanner, or a networking devicesuch as a switch or router, e.g., through a network adapter.

The computing device 400 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as astandard server 420, or multiple times in a group of such servers. Itmay also be implemented as part of a rack server system 424. Inaddition, it may be implemented in a personal computer such as a laptopcomputer 422. Alternatively, components from computing device 400 may becombined with other components in a mobile device (not shown), such ascomputing device 450. Each of such devices may contain one or more ofcomputing devices 400, 450, and an entire system may be made up ofmultiple computing devices 400, 450 communicating with each other.

Computing device 450 includes a processor 452, memory 464, aninput/output device such as a display 454, a communication interface466, and a transceiver 468, among other components. The computing device450 may also be provided with a storage device, such as a micro-drive orother device, to provide additional storage. Each of the components 450,452, 464, 454, 466, and 468, are interconnected using various buses, andseveral of the components may be mounted on a common motherboard or inother manners as appropriate.

The processor 452 can process instructions for execution within thecomputing device 450, including instructions stored in the memory 464.The processor may also include separate analog and digital processors.The processor may provide, for example, for coordination of the othercomponents of the computing device 450, such as control of userinterfaces, applications run by computing device 450, and wirelesscommunication by computing device 450.

Processor 452 may communicate with a user through control interface 458and display interface 456 coupled to a display 454. The display 454 maybe, for example, a TFT LCD display or an OLED display, or otherappropriate display technology. The display interface 456 may compriseappropriate circuitry for driving the display 454 to present graphicaland other information to a user. The control interface 458 may receivecommands from a user and convert them for submission to the processor452. In addition, an external interface 462 may be provided incommunication with processor 452, so as to enable near areacommunication of computing device 450 with other devices. Externalinterface 462 may provide, for example, for wired communication (e.g.,via a docking procedure) or for wireless communication (e.g., viaBluetooth® or other such technologies).

The memory 464 stores information within the computing device 450. Inone implementation, the memory 464 is a computer-readable medium. In oneimplementation, the memory 464 is a volatile memory unit or units. Inanother implementation, the memory 464 is a non-volatile memory unit orunits. Expansion memory 474 may also be provided and connected tocomputing device 450 through expansion interface 472, which may include,for example, a subscriber identification module (SIM) card interface.Such expansion memory 474 may provide extra storage space for computingdevice 450, or may also store applications or other information forcomputing device 450. Specifically, expansion memory 474 may includeinstructions to carry out or supplement the processes described above,and may include secure information also. Thus, for example, expansionmemory 474 may be provide as a security module for computing device 450,and may be programmed with instructions that permit secure use ofcomputing device 450. In addition, secure applications may be providedvia the SIM cards, along with additional information, such as placingidentifying information on the SIM card in a non-hackable manner.

The memory may include for example, flash memory and/or MRAM memory, asdiscussed below. In one implementation, a computer program product istangibly embodied in an information carrier. The computer programproduct contains instructions that, when executed, perform one or moremethods, such as those described above. The information carrier is acomputer- or machine-readable medium, such as the memory 464, expansionmemory 474, or memory on processor 452.

Computing device 450 may communicate wirelessly through communicationinterface 466, which may include digital signal processing circuitrywhere necessary. Communication interface 466 may provide forcommunications under various modes or protocols, such as GSM voicecalls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA, CDMA2000, orGPRS, among others. Such communication may occur, for example, throughtransceiver 468 (e.g., a radio-frequency transceiver). In addition,short-range communication may occur, such as using a Bluetooth®, WiFi,or other such transceiver (not shown). In addition, GPS receiver module470 may provide additional wireless data to computing device 450, whichmay be used as appropriate by applications running on computing device450.

Computing device 450 may also communicate audibly using audio codec 460,which may receive spoken information from a user and convert it tousable digital information. Audio codec 460 may likewise generateaudible sound for a user, such as through a speaker, e.g., in a handsetof computing device 450. Such sound may include sound from voicetelephone calls, may include recorded sound (e.g., voice messages, musicfiles, etc.) and may also include sound generated by applicationsoperating on computing device 450.

The computing device 450 may be implemented in a number of differentforms, as shown in the figure. For example, it may be implemented as acellular telephone 480. It may also be implemented as part of asmartphone 482, personal digital assistant, or other mobile device.

Various implementations of the systems and techniques described here canbe realized in digital electronic circuitry, integrated circuitry,specially designed ASICs (application specific integrated circuits),computer hardware, firmware, software, and/or combinations thereof.These various implementations can include implementation in one or morecomputer programs that are executable and/or interpretable on aprogrammable system including at least one programmable processor, whichmay be special or general purpose, coupled to receive data andinstructions from, and to transmit data and instructions to, a storagesystem, at least one input device, and at least one output device.

These computer programs (also known as programs, software, softwareapplications or code) include machine instructions for a programmableprocessor, and can be implemented in a high-level procedural and/orobject-oriented programming language, and/or in assembly/machinelanguage. Other programming paradigms can be used, e.g., functionalprogramming, logical programming, or other programming. As used herein,the terms “machine-readable medium” “computer-readable medium” refers toany computer program product, apparatus and/or device (e.g., magneticdiscs, optical disks, memory, Programmable Logic Devices (PLDs)) used toprovide machine instructions and/or data to a programmable processor,including a machine-readable medium that receives machine instructionsas a machine-readable signal. The term “machine-readable signal” refersto any signal used to provide machine instructions and/or data to aprogrammable processor.

To provide for interaction with a user, the systems and techniquesdescribed here can be implemented on a computer having a display device(e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor)for displaying information to the user and a keyboard and a pointingdevice (e.g., a mouse or a trackball) by which the user can provideinput to the computer. Other kinds of devices can be used to provide forinteraction with a user as well; for example, feedback provided to theuser can be any form of sensory feedback (e.g., visual feedback,auditory feedback, or tactile feedback); and input from the user can bereceived in any form, including acoustic, speech, or tactile input.

The systems and techniques described here can be implemented in acomputing system that includes a back end component (e.g., as a dataserver), or that includes a middleware component (e.g., an applicationserver), or that includes a front end component (e.g., a client computerhaving a graphical user interface or a Web browser through which a usercan interact with an implementation of the systems and techniquesdescribed here), or any combination of such back end, middleware, orfront end components. The components of the system can be interconnectedby any form or medium of digital data communication (e.g., acommunication network). Examples of communication networks include alocal area network (“LAN”), a wide area network (“WAN”), and theInternet.

The computing system can include clients and servers. A client andserver are generally remote from each other and typically interactthrough a communication network. The relationship of client and serverarises by virtue of computer programs running on the respectivecomputers and having a client-server relationship to each other.

While this specification contains many specific implementation details,these should not be construed as limitations on the scope of anyinventions or of what may be claimed, but rather as descriptions offeatures specific to particular implementations of particularinventions. Certain features that are described in this specification inthe context of separate implementations can also be implemented incombination in a single implementation. Conversely, various featuresthat are described in the context of a single implementation can also beimplemented in multiple implementations separately or in any suitablesubcombination. Moreover, although features may be described above asacting in certain combinations and even initially claimed as such, oneor more features from a claimed combination can in some cases be excisedfrom the combination, and the claimed combination may be directed to asubcombination or variation of a subcombination.

Similarly, while operations are depicted in the drawings in a particularorder, this should not be understood as requiring that such operationsbe performed in the particular order shown or in sequential order, orthat all illustrated operations be performed, to achieve desirableresults. In certain circumstances, multitasking and parallel processingmay be advantageous. Moreover, the separation of various systemcomponents in the implementations described above should not beunderstood as requiring such separation in all implementations, and itshould be understood that the described program components and systemscan generally be integrated together in a single software product orpackaged into multiple software products.

Thus, particular implementations of the subject matter have beendescribed. Other implementations are within the scope of the followingclaims. In some cases, the actions recited in the claims can beperformed in a different order and still achieve desirable results. Inaddition, the processes depicted in the accompanying figures do notnecessarily require the particular order shown, or sequential order, toachieve desirable results. In certain implementations, multitasking andparallel processing may be advantageous.

What is claimed is:
 1. (canceled)
 2. A method, comprising: receiving,from a content sponsor, selection criteria for presenting a firstcontent item, including receiving an indication that the first contentitem is to be presented when a specified group of related entities arepresented in a search results page; storing a reference to the firstcontent item and the selection criteria; after the storing, receiving aquery including one or more terms or phrases that do not identify thespecified group of related entities; determining, by one or moreprocessors, that the one or more terms or phrases refer to a firstentity; determining, using one or more processors, that the first entityis included in the specified group of related entities and that aportion of the group of related entities are being provided forpresentation on a responsive search results page; identifying, from aninventory of content items, the first content item as eligible forpresentation based on the determination that the portion of the group ofrelated entities are being provided for presentation on the responsivesearch results page; and dynamically arranging, on the responsive searchresult page, a presentation of the portion of the related entities toposition the first content item more prominently or to visuallyemphasize the first content item based on a bid by the content sponsor.3. The method of claim 2 wherein the specified group of related entitiesis named, and wherein the method further comprises providing to thecontent sponsor an interface for enabling a selection of the specifiedgroup of related entities from a plurality of available collection ofentities, and receiving from content sponsor an explicit designation ofthe specified group of related entities by name to be used as theselection criteria for delivery of the first content item.
 4. The methodof claim 3 wherein the explicit designation includes a designation ofthe specified group of related entities and an amount of a bid to beused when evaluating the first content item with other eligible contentitems in an auction or reservation selection system.
 5. The method ofclaim 2 wherein determining that the first entity is included in thespecified group of related entities includes determining when the one ormore terms or phrases indicates a collection of entities and identifyingthe collection entities as the specified group of related entities. 6.The method of claim 2 wherein determining that the first entity isincluded in the specified group of related entities includes determiningwhen the one or more terms or phrases indicates the first entity,locating a collection that includes the first entity and identifying thecollection that includes the first entity as the specified group ofrelated entities.
 7. The method of claim 2 wherein determining that thefirst entity is included in the specified group of related entitiesincludes: determining when the one or more terms or phrases indicates acollection of entities and identifying the collection as the specifiedgroup of related entities; or determining when the one or more terms orphrases indicates the first entity, locating a collection that includesthe first entity and identifying the collection that includes the firstentity as the specified group of related entities.
 8. The method ofclaim 2 further comprising: determining a plurality of collections ofentities based at least in part on the one or more terms or phrases; andselecting one of the plurality of collections of entities as thespecified group of related entities based at least in part on apredetermined selection criteria.
 9. The method of claim 8 wherein thepredetermined selection criteria is selected from the group comprising amost popular collection, a most likely collection, or a most relevantcollection.
 10. The method of claim 2 wherein the specified group ofrelated entities includes a plurality of elements that are displayableon a device for presentation to a user, and wherein the first contentitem is presented in a slot that precedes, succeeds or is interposedwith the elements of the specified group of related entities.
 11. Themethod of claim 2 wherein the first content item is an advertisement andthe specified group of related entities is a group of entities that aredistinct from the search results.
 12. The method of claim 2 furthercomprising: presenting an interface to a content sponsor for a campaignthat is associated with distribution of the first content item, theinterface including a control for indicating one or more selectioncriteria for use in determining when to identify the first content itemas an eligible content item for consideration for delivery in responseto a received request for a content item; receiving a selection usingthe control of a collection, the collection representing a group ofentities that share at least one common characteristic, the collectiondesignation specifying when to present the first content item responsiveto requests including requiring at least a portion of the collection bepresented along with the first content item; and storing the campaignincluding storing an association between the selected collection and thefirst content item.
 13. The method of claim 12 further comprisingverifying that the first content item qualifies as being included in thespecified group of related entities else disallowing the selection touse the specified group of related entities as selection criteria forthe campaign.
 14. A computer program product embodied in anon-transitive computer-readable medium including instructions, thatwhen executed, cause one or more processors to: receive, from a contentsponsor, selection criteria for presenting a first content item,including receiving an indication that the first content item is to bepresented when a specified group of related entities are presented in asearch results page; store a reference to the first content item and theselection criteria; after the storing, receive a query including one ormore terms or phrases that do not identify the specified group ofrelated entities; determine, by one or more processors, that the one ormore terms or phrases refer to a first entity; determine, using one ormore processors, that the first entity is included in the specifiedgroup of related entities and that a portion of the group of relatedentities are being provided for presentation on a responsive searchresults page; identify, from an inventory of content items, the firstcontent item as eligible for presentation based on the determinationthat the portion of the group of related entities are being provided forpresentation on the responsive search results page; and dynamicallyarrange, on the responsive search result page, a presentation of theportion of the related entities to position the first content item moreprominently or to visually emphasize the first content item based on abid by the content sponsor.
 15. The computer program product of claim 14wherein the specified group of related entities is named, and whereinthe instructions further include instructions that cause the one or moreprocessors to provide to the content sponsor an interface for enabling aselection of the specified group of related entities from a plurality ofavailable collection of entities, and receive from content sponsor anexplicit designation of the specified group of related entities by nameto be used as the selection criteria for delivery of the first contentitem.
 16. The computer program product of claim 15 wherein the explicitdesignation includes a designation of the specified group of relatedentities and an amount of a bid to be used when evaluating the firstcontent item with other eligible content items in an auction orreservation selection system.
 17. The computer program product of claim14 wherein determining that the first entity is included in thespecified group of related entities includes determining when the one ormore terms or phrases indicates a collection of entities and identifyingthe collection entities as the specified group of related entities. 18.A system comprising: an entity engine that evaluates terms and phrasesin a received query to identify associated collections of entities; arelated content identification engine that identifies eligible contentitems from an inventory of content items, wherein each eligible contentitem identified is associated with selection criteria, includingcriteria specifying, or associated with, a group of related entities andthe received query; a request handler that handles requests for contentreceived by a content management system, the content management systemselecting and providing content in response to requests for content; oneor more processors; and one or more memory elements includinginstructions that, when executed, cause the one or more processors to:receive, from a content sponsor, selection criteria for presenting afirst content item, including receiving an indication that the firstcontent item is to be presented when a specified group of relatedentities are presented in a search results page; store a reference tothe first content item and the selection criteria; after the storing,receive a query including one or more terms or phrases that do notidentify the specified group of related entities; determine, by one ormore processors, that the one or more terms or phrases refer to a firstentity; determine, using one or more processors, that the first entityis included in the specified group of related entities and that aportion of the group of related entities are being provided forpresentation on a responsive search results page; identify, from aninventory of content items, the first content item as eligible forpresentation based on the determination that the portion of the group ofrelated entities are being provided for presentation on the responsivesearch results page; and dynamically arrange, on the responsive searchresult page, a presentation of the portion of the related entities toposition the first content item more prominently or to visuallyemphasize the first content item based on a bid by the content sponsor.19. The system of claim 18 wherein the specified group of relatedentities is named, and wherein the instructions further includeinstructions that cause the one or more processors to provide to thecontent sponsor an interface for enabling a selection of the specifiedgroup of related entities from a plurality of available collection ofentities, and receive from content sponsor an explicit designation ofthe specified group of related entities by name to be used as theselection criteria for delivery of the first content item.
 20. Thesystem of claim 19 wherein the explicit designation includes adesignation of the specified group of related entities and an amount ofa bid to be used when evaluating the first content item with othereligible content items in an auction or reservation selection system.21. The system of claim 18 wherein determining that the first entity isincluded in the specified group of related entities includes determiningwhen the one or more terms or phrases indicates a collection of entitiesand identifying the collection entities as the specified group ofrelated entities.